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DETAILED ACTION 
Drawings 

1. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because a) 
reference characters "192" in Fig. 1 and "163" on page 13 of the specification have both been 
used to designate a camera, b) reference characters "193" in Fig. 1 and "164" on page 13 of the 
specification have both been used to designate a sequence of images, and c) reference characters 
"194" in Fig. 1 and "165" on page 13 of the specification have both been used to designate a 
camera interface. A proposed drawing correction or corrected drawings are required in reply to 
the Office action to avoid abandonment of the application. The objection to the drawings will 
not be held in abeyance. 

Claim Objections 

2. The following quotation of 37 CFR § 1.75(a) is the basis of objection: 

(a) The specification must conclude with a claim particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention or discovery. 

3. Claims 1-21 are objected to under 37 CFR § 1.75 as failing to particularly point out and 
distinctly claim the subject matter which the applicant regards as his invention or discovery. 

The claims are replete with antecedent problems. The following objections are merely 
exemplary of the overall problems with the claims. The examiner requests a complete review of 
the claim to address additional problems that are not explicitly addressed below. 

Regarding claim 1, the claim recites the limitations "the camera pose parameters" in line 
2, "the three successive images of each triplet of images" in line 6, and "the camera coordinate 
system" in line 11. There is insufficient antecedent basis for these limitations in the claims. 
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Regarding claim 2, the claim recites the limitations "the action" in line 3 and "the sum of 
squared differences between the image coordinates" in lines 5 and 7. There is insufficient 
antecedent basis for these limitations in the claims. 

Regarding claim 3, the claim recites the limitation "the 3D world coordinates" in lines 3 
and 7. There is insufficient antecedent basis for these limitations in the claims. 

Regarding claim 5, the claim recites the limitations "the locations of each pair of triple 
matching points" in line 5 and "the fundamental matrix" in line 6. There is insufficient 
antecedent basis for these limitations in the claims. 

Regarding claim 9, the claim recites the limitation "the camera projection matrix" in lines 
1-2 and "the camera coordinate system" in line 12. There is insufficient antecedent basis for 
these limitations in the claims. 

Regarding claim 14, the claim recites the limitation "the camera pose parameters" in line 
2 and "the camera coordinate system" in line 8. There is insufficient antecedent basis for these 
limitations in the claims. 

Regarding claim 15, the claim recites the limitation "the camera pose parameters" in lines 
1-2 and "the camera coordinate system" in line 7. There is insufficient antecedent basis for these 
limitations in the claims. 

As stated above, these objections are merely exemplary and the examiner requests a 
complete review of the claims for any additional problems. 

Claims 3 and 4 also call for the limitation "of an either triple or dual matching point" in 
line 2 of each claim. This limitation is grammatically awkward. The examiner suggests 
amending this limitation to read "of either a triple or dual matching point". 
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Claims 6-8, 10-13, and 16-21 are objected to as depending from objected to claims. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the T Jnited States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1(2) of such treaty in the English language. 

5. Claims 1, 9, 10, 12, 14-16, and 19 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Davison et al. (U.S. Patent No. 6,668,082 Bl). 

As applied to claim 1, Davison et al. disclose a computer-implemented incremental 
motion estimation process for estimating the camera pose parameters associated with each of a 
sequence of images of a scene, comprising using a computer to perform the following process 
actions (see Fig. 1 : As can be seen in the figure, CPU 4 is used to perform all of the process 
actions.): establishing a set of triple matching points that depict the same point in the scene 
across the three successive images of each triplet of images that can be formed from the 
sequence (see column 22, lines 39-42: The reference describes that the CPU 4 generates a list of 
points which are matched across all three images in the triple of images (i.e. establishing a set of 
triple matching points that depict the same point in the scene across the three successive images 
of each triplet of images that can be formed from the sequence).); establishing a set of dual 
matching points that depict the same point in the scene across the latter two images of each 
triplet of images that can be formed from the sequence (see column 22, lines 39-42: The 
reference describes that points are established that match between each of the images in a triple 
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of images. Therefore, these points also match across the latter two images of the triple of images 
and can be considered dual matching points.); defining the camera coordinate system associated 
with the first image in the sequence as coinciding with a 3D world coordinate system for the 
scene depicted in the sequence of images (see column 22, lines 47-51: The reference describes 
that a coordinate system in mm is established having an origin at the camera optical center (i.e. 
defining the camera coordinate system) for each of the images in the triple of images. Therefore, 
it is defined for the first image. The reference also describes that this coordinate system is 
converted from the pixel coordinate system (i.e. the world coordinate system).); estimating the 
camera pose parameters associated with the second image in the sequence (see column 23, lines 
18-23: The reference describes that the camera transformations (i.e. camera pose parameters) are 
calculated (i.e. estimated) for the first pair of images and the second pair of images in the 
sequence. Both the first pair of images and the second pair of images include the second image.); 
and for each successive triplet of images formable from the sequence starting with the first three 
images (see column 9, line 67 - column 10, line 1: The reference describes that the processing 
steps are repeated for the next triple of images.), estimating the camera pose parameters 
associated with the third image of each triplet from previously-ascertained camera pose 
parameters associated with the first two images in the triplet, as well as the dual and triple 
matching points of the triplet (see column 23, lines 18-23: The reference describes that the 
camera pose parameters (i.e. camera pose parameters) for the second pair of images using 
information from the first pair of images (i.e. from previously-ascertained camera pose 
parameters associated with the first two images in the triplet) and the "triple" points (i.e. the dual 
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and triple matching points of the triplet). The second pair of images includes the third image in 
the triple of images.)- 

As applied to claim 9, Davison et al. disclose a local bundling adjustment system for 
estimating the camera projection matrix associated with each of a sequence of images of a scene, 
comprising: a general purpose computing device (see Fig. 1 : As can be seen in the figure, CPU 4 
is used to perform all of the process actions.); a computer program comprising program modules 
executable by the computing device (see column 6, lines 16-18: The reference describes that 
operating instructions for the CPU 4 are input from a removable data storage medium using disk 
drive 8.), wherein the computing device is directed by the program modules of the computer 
program to, input the sequence of images (see Fig. 1 : Reference numeral 12, referring to a 
camera for inputting a sequence of images.):, identify points in each image in the sequence of 
images (see column 22, lines 21-26: The reference describes that the CPU generates a list of 
matched points in each image in the sequence of images.); determine which of the identified 
points depict the same point of the scene so as to be matching points (see column 22, lines 39-42: 
The reference describes that the CPU 4 determines a list of points which are matched across all 
three images in the triple (i.e. determine which of the identified points depict the same point of 
the scene so as to be matching points).), define the camera coordinate system associated with the 
first image in the sequence as coinciding with a 3D world coordinate system for the scene 
depicted in the sequence of images (see column 22, lines 47-51: The reference describes that a 
coordinate system in mm is established having an origin at the camera optical center (i.e. 
defining the camera coordinate system) for each of the images in the triple of images. Therefore, 
it is defined for the first image. The reference also describes that this coordinate system is 
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converted from the pixel coordinate system (i.e. the world coordinate system).); compute the 
camera projection matrix associated with the second image in the sequence (see column 23, lines 
18-23: The reference describes that the camera transformations (i.e. camera projection matrix) 
are calculated (i.e. computed) for the first pair of images and the second pair of images in the 
sequence. Both the first pair of images and the second pair of images include the second image.), 
and for each successive triplet of images that can be formed from the sequence starting with the 
first three images (see column 9, line 67 - column 10, line 1: The reference describes that the 
processing steps are repeated for the next triple of images.), estimate the camera projection 
matrix of the third image of each triplet from previously-ascertained camera projection matrices 
associated with the first two images in the triplet as well as points determined to be matching 
points across all three images of the of the triplet and points determined to be matching points 
across at least one pair of images in the triplet (see column 23, lines 18-23: The reference 
describes that the camera pose parameters (i.e. camera projection matrices) for the second pair of 
images using information from the first pair of images (i.e. from previously-ascertained camera 
projection matrices associated with the first two images in the triplet) and the "triple" points (i.e. 
the matching points across all three images of the of the triplet and points determined to be 
matching points across at least one pair of images in the triplet). The second pair of images 
includes the third image in the triple of images.). 

As applied to claim 10, Davison et al. disclose that the program module for determining 
which of the identified points depict the same point of the scene so as to be matching points 
comprises sub-modules for: establishing a set of triple matching points that depict the same point 
in the scene across the three successive images of each triplet of images that can be formed from 
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the sequence (see column 22, lines 39-42: The reference describes that the CPU 4 generates a list 
of points which are matched across all three images in the triple of images (i.e. establishing a set 
of triple matching points that depict the same point in the scene across the three successive 
images of each triplet of images that can be formed from the sequence).); and establishing a set 
of dual matching points that depict the same point in the scene across the latter two images of 
each triplet of images that can be formed from the sequence (see column 22, lines 39-42: The 
reference describes that points are established that match between each of the images in a triple 
of images. Therefore, these points also match across the latter two images of the triple of images 
and can be considered dual matching points.). 

As applied to claim 12, Davison et al. disclose that the program module for determining 
which of the identified points depict the same point of the scene so as to be matching points 
comprises sub-modules for: establishing a set of triple matching points that depict the same point 
in the scene across the three successive images of each triplet of images that can be formed from 
the sequence (see column 22, lines 39-42: The reference describes that the CPU 4 generates a list 
of points which are matched across all three images in the triple of images (i.e. establishing a set 
of triple matching points that depict the same point in the scene across the three successive 
images of each triplet of images that can be formed from the sequence).); establishing a first set 
of dual matching points that depict the same point in the scene across the latter two images of 
each triplet of images that can be formed from the sequence (see column 22, lines 39-42: The 
reference describes that points are established that match between each of the images in a triple 
of images. Therefore, these points also match across the latter two images of the triple of images 
and can be considered dual matching points.); and establishing a second set of dual matching 
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points that depict the same point in the scene in the first and third images of each triplet of 
images that can be formed from the sequence (see column 22, lines 39-42: The reference 
describes that points are established that match between each of the images in a triple of images. 
Therefore, these points also match across the first and third images of the triple of images and 
can be considered dual matching points.). 

As applied to claim 14, which merely calls for a computer-readable medium having 
computer-executable instructions for performing the process performed by the system of claim 9, 
Davison et al. discloses such a computer-readable medium (see column 6, lines 16-18: The 
reference describes that operating instructions for the CPU 4 may be input from a removable data 
storage medium using disk drive 8.). 

As applied to claims 15, 16, and 19, which merely call for the process performed by the 
system in claims 9, 10, and 12, since Davison et al. disclose the system, then the process 
performed by the system is also disclosed. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Davison et al. (U.S. Patent No. 6,668,082 Bl) and Fitzgibbon et al. (the article 
titled "Automatic Camera Recovery for Closed or Open Image Sequences"). The arguments as to 
the relevance of Davison et al. in the rejection of claim 1 above are incorporated herein. 
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Claim 7 calls for the process action of estimating the camera pose parameters associated 
with the second image in the sequence, to comprise the action of using a two-view 
structure-from-motion technique. 

Davison et al. does not disclose the use of such a technique for estimating the camera 
pose parameters. 

However, Fitzgibbon et al, in the same field of endeavor of image processing and the 
same problem solving area of feature extraction, disclose the use of such a two-view 
structure-from-motion technique (see section 3.3, heading "Two-view overlap": The reference 
describes the use of a two-view overlap technique (i.e. two-view structure-from-motion 
technique) for estimating camera pose parameters.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Davison et al. by adding the use of a two-view 
structure-from-motion technique for estimating camera pose parameters as taught in Fitzgibbon 
et al. because the use of such a technique "has the advantage that shorter tracks are need - four 
views rather than five - and that any false inliers to the trifocal tensors may be identified because 
their tracks are inconsistent" (see Fitzgibbon: section 3.3, heading "Two-view overlap"). 
Therefore, the system can estimate the camera pose parameters using less information - four 
views rather than five - thus increasing processing speed. 

Claim 8 calls for the process action of estimating the camera pose parameters associated 
with the second image in the sequence, to comprise the action of applying a global bundle 
alignment technique to the first three images of the sequence of images. 
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Davison et al. does not disclose the use of a global bundle alignment technique to 
estimate camera pose parameters. 

However, Fitzgibbon et al. 3 in the same field of endeavor of image processing and the 
same problem solving area of feature extraction, disclose the use of such a global bundle 
alignment technique (see section A, titled "Bundle Adjustment": The reference describes the use 
of bundle adjustment for estimating camera pose.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Davison et al. by adding the use of a global bundle alignment 
technique to estimate camera pose parameters as taught in Fitzgibbon et al. because the use of 
such a technique gives the system "the ability to quickly compute maximum likelihood estimates 
of cameras and structure (see Fitzgibbons et al.: section A, titled "Bundle Adjustment"). 

Allowable Subject Matter 
8. Claims 2-6, 11, 13, 17, 18, 20, and 21 would be allowable if rewritten to overcome the 
objections under 37 C.F.R. 1.75 (a) set forth in this Office action and to include all of the 
limitations of the base claim and any intervening claims. 

Claim 2, which is representative of claims 11, 13, 17, and 20 calls for the action of 
computing the camera pose parameters associated with the third image in the triplet under 
consideration which will result in a minimum value for the sum of the squared differences 
between the image coordinates of each identified triple match point and its corresponding 
predicted image coordinates in each of the three images in the triplet, added to the sum of the 
squared differences between the image coordinates of each identified dual match point and its 
corresponding predicted image coordinates in each of the latter two images in the triplet. These 
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claims describe "a specialized procedure for speeding up the processing associated with the 
present local bundle adjustment technique" (see applicant's specification: page 19, lines 4-6). 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Kumar et al. (U.S. Patent No. 6,597,818 B2) is pertinent in that the reference describes a 
system for mapping between image coordinates and geo-spatial coordinates (i.e. camera 
coordinates). 

Shum et al. (the article titled "Efficient Bundle Adjustment with Virtual Key Frames: A 
Hierarchical Approach to Multi-frame Structure from Motion") is pertinent in that the reference 
discloses the use of bundle adjustment for the estimation of motion. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J. Miller whose telephone number is (703) 306-4142. The 
examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 



Application/Control Number: 09/884,355 



Page 13 



Art Unit: 2621 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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